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TITLE OF THE INVENTION 



IMAGE SENSING APPARATUS 

BACKGROUND OF THE INVENTION 

FIELD OF THE INVENTION 

The present invention relates to an image sensing 
apparatus for sensing a still image and/or moving image. 

DESCRIPTION OF THE RELATED ART 

Conventionally, image sensing apparatuses such as 
electronic cameras and the like, which record/play back 
still images and moving images sensed by a solid-state 
image sensing element such as a CCD or the like using 
memory cards having solid-state memory elements as 
recording media, have already been commercially 
available . 

Furthermore, another electronic still camera 
senses images by selectively using one of an aperture 
(Av) priority image sensing mode for automatically 
increasing/decreasing the shutter speed to obtain an 
appropriate exposure value in correspondence with an 
arbitrarily selected aperture value, and a shutter 
speed (Tv) priority mode for automatically 
increasing/decreasing the aperture value to obtain an 



appropriate exposure value in correspondence with an 
arbitrarily selected shutter speed. 

Moreover, still another electronic still camera 
comprises an AEB {auto exposure bracketing) image 
5 sensing mode that automatically brackets three frames 
with a standard exposure value, overexposure value, and 
underexposure value during single image sensing 
operation. Such cameras enable the photographer to 
easily obtain an appropriate exposure value upon image 

10 sensing under complicated light conditions or upon 

image sensing using an electronic camera that comprises 
an image sensing element with a narrow latitude. 

Upon image sensing using a solid-state image 
sensing element such as a CCD or the like, a dark noise 

15 correction process is done by computations using dark 
image data which is read out after charge accumulation 
in the same manner as in actual image sensing while the 
image sensing element is not exposed, and actually 
sensed image data which is read out after charge 

20 accumulation while the image sensing element is exposed. 
With this process, sensed image data can be 
corrected for image quality deterioration such as pixel 
omission or the like caused by dark current noise 
produced by the image sensing element and small 

25 scratches unique to the image sensing element, thus 
obtaining high-quality sensed image data. 



Especially, since dark current noise increases 
with increasing charge accumulation time and 
temperature rise of the image sensing element, if 
exposure for a long period of time (seconds) or at high 
5 temperature is done, a great image quality improvement 
effect can be obtained, and the dark noise correction 
process is a function useful for the electronic camera 
users . 

In this manner, since dark current noise 

10 increases with increasing charge accumulation time and 
temperature rise of the image sensing element, the 
charge accumulation time of the image sensing element 
for capturing the dark image data must be determined in 
correspondence with the shutter speed (Tv value) 

15 obtained by exposure measurement. 

On the other hand, when AEB image sensing is done 
in the aperture (Av) priority image sensing mode, the 
shutter speed must be changed for each frame upon 
automatic bracketing for three frames. 

20 Therefore, in an image sensing apparatus such as 

an electronic camera, when the shutter speed has 
changed, a new charge accumulation time is set. When 
the charge accumulation time has been changed, the dark 
image must be re-captured. That is, upon executing 

25 automatic sequential exposure image sensing for three 



frames, the dark image must be re-captured every actual 
image sensing. 

For this reason, when continuous-shot AEB image 
sensing is done in the aperture (Av) priority image 
5 sensing mode, the dark image must be captured three 
times upon sensing each of frames to be continuously 
sensed, and the image sensing interval for each frame 
upon continuous shot image sensing is prolonged by the 
dark image sensing time, and the shutter release time 
10 lag becomes longer by the dark image sensing time, thus 
making the user miss a shutter chance or reducing the 
continuous shot frame speed. 



SUMMARY OF THE INVENTION 

15 ' One aspect of the present invention is to provide 

an apparatus, which comprises an image sensing device, 
a signal processing device for performing a first image 
sensing operation for making the image sensing device 
perform an image sensing operation in an exposure state 

20 to obtain a sensed image signal, and a second image 

sensing operation for making the image sensing device 
perform an image sensing operation in accordance with 
the first image sensing operation in a non-exposure 
state to obtain a sensed image signal, and processing 

25 the sensed image signal obtained by the first image 

sensing operation by the sensed image signal obtained 



by the second image sensing operation, wherein the 
signal processing device changes method of the second 
image sensing operation to the first image sensing 
operation in response to the instruction of the" 
5 instruction device, and an instruction device for 

instructing to execute a predetermined plurality times 
of image sensing operations with different image 
sensing times of the first image sensing operation, 
whereby the apparatus can prevent a shutter chance from 
10 being missed, and can prevent low continuous shot frame 
speed. 

Other aspects of the present invention will be 
apparent from the following description taken in 
conjunction with the accompanying drawings, in which 
15 like reference characters designate the same or similar 
parts throughout the figures thereof. 



BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block diagram showing the arrangement 
20 of an image sensing apparatus according to an 
embodiment of the present invention; 

Fig. 2 is a flow chart of a main routine showing 
the control sequence of the image processing apparatus; 
Fig. 3 is a flow chart of the main routine 
25 showing the control sequence of the image processing 
apparatus and continued from Fig. 2; 
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Fig. 4 is a flow chart of the main routine 
showing the control sequence of the image processing 
apparatus and continued from Fig. 3; 

Fig. 5 is a flow chart of the main routine 
5 showing the control sequence of the image processing 
apparatus and continued from Fig. 4 ; 

Fig. 6 is a flow chart of the main routine 
showing the control sequence of the image processing 
apparatus and continued from Fig. 5; 
10 Fig. 7 is a flow chart showing the detailed 

sequence of the distance measurement /photometry process 
in step S134 in Fig. 4; 

Fig. 8 is a flow chart showing the detailed 
sequence of the image sensing process in step S164 in 
15 Fig. 4; 

Fig. 9 is a flow chart showing the detailed 
sequence of the dark capture process in step S140 in 
Fig. 4 and in step S167 in Fig. 5; and 

Fig. 10 is a timing chart showing the flow of 
20 image sensing operation in the image sensing apparatus. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
A preferred embodiment of the present invention 
will be described hereinafter with reference to the 
25 accompanying drawings. 
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Fig. 1 is a block diagram showing the arrangement 
of an image sensing apparatus according to an 
embodiment of the present invention. 

Referring to Fig. 1, reference numeral 100 
5 denotes an image processing apparatus. 

Reference numeral 12 denotes a shutter for 
controlling the exposure amount on an image sensing 
element 14; and 14, an image sensing element for 
converting an optical image into an electrical signal. 
10 Light rays that have entered a lens 310 are 

guided onto the image sensing element 14 via a stop 312, 
lens mounts 306 and 106, a mirror 130, and the shutter 
12 by a single-lens reflex system, and form an optical 
image on the image sensing element 14. 
15 Reference numeral 16 denotes an A/D converter for 

converting an analog signal output from the image 
sensing element 14 into a digital signal. 

Reference numeral 18 denotes a timing generation 
circuit for supplying clock signals and control signals 
20 to the image sensing element 14, the A/D converter 16, 
and a D/A converter 26. The timing generation circuit 
18 is controlled by a memory control circuit 22 and 
system control circuit 50. 

Reference numeral 20 denotes an image processing 
25 circuit for performing a predetermined pixel 

interpolation process and color conversion process for 



data from the A/D converter 16 or the memory control 
circuit 22. 

Also, the image processing circuit 20 can execute 
a TTL (through the lens) AF (auto-focus) process, AE 
5 (auto-exposure) process, and EF (flash light control) 
process. In these processes, the image processing 
circuit 20 makes predetermined computations using 
sensed image data as needed, and the system control 
circuit 50 controls an exposure (shutter) control 
10 circuit 4 0 and distance measurement circuit 42 on the 
basis of the obtained computation result. 

Furthermore, the image processing circuit 20 
makes predetermined computations using sensed image 
data, and executes a TTL AWB (auto white balance) 
15 process on the basis of the obtained computation result. 
Since this embodiment comprises the distance 
measurement circuit 42 and a photometry circuit 46 
dedicated to those processes, the AF, AE, and EF 
processes may be done using the distance measurement 
20 circuit 42 and photometry circuit 4 6 in place of those 
using the image processing circuit 20. 

Alternatively, the AF, AE, and EF processes may 
be done using the distance measurement circuit 42 and 
photometry circuit 46 in addition to those using the 
25 image processing circuit 20. 



Reference numeral 22 denotes a memory control 
circuit which controls the A/D converter 16, the timing 
generation circuit 18, the image processing circuit 20, 
an image display memory 24, the D/A converter 26, a 
5 memory 30, and a compression/expansion circuit 32. 

Data output from the A/D converter 16 is written 
in the image display memory 24 or memory 30 via the 
image processing circuit 20 and memory control circuit 
22 or via the memory control circuit 22 alone. 
10 Reference numeral 24 denotes an image display 

memory; 26, a D/A converter; and 28, an image display 
unit comprising, e.g., a TFT LCD or the like. Display 
image data written in the image display memory 24 is 
displayed on the image display unit 28 via the D/A 
15 converter 26. 

When sensed image data is displayed using the 
image display unit 28 as needed, an electronic 
viewfinder function can be implemented. 

The image display unit 28 can turn on/off its 
20 display in accordance with an instruction from the 
system control circuit 50. When the display is OFF, 
great power savings of the image processing apparatus 
100 can be achieved. 

Reference numeral 30 denotes a memory for storing 
25 sensed still or moving images. The memory 30 has a 
sufficient memory size capable of storing a 



predetermined number of still images or a moving image 
for a predetermined period of time. Hence, even in a 
continuous shot mode or panorama mode for continuously 
sensing a plurality of still images, a large number of 
5 images can be written in the memory 30 at high speed. 

Also, the memory 30 can be used as a work area of 
the system control circuit 50. 

Reference numeral 32 denotes a 
compression/expansion circuit for compressing/expanding 

10 image data by the adaptive discrete cosine transform 

(ADCT) , or the like. The compression/expansion circuit 
32 loads an image stored in the memory 30, compresses 
or expands it, and writes the compressed or expanded 
data in the memory 30. 

15 Reference numeral 4 0 denotes a shutter control 

circuit for controlling the shutter 12 on the basis of 
photometry information from the photometry circuit 46 
in cooperation with a stop control circuit 340 that 
controls the stop 312. 

20 Reference numeral 42 denotes a distance 

measurement circuit used to execute the AF process. 
Light rays that have entered the lens 310 are guided to 
the distance measurement circuit 42 via the stop 312, 
the lens mounts 306 and 106, the mirror 130, and a 

25 distance measurement sub-mirror (not shown) by the 
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single-lens reflex system, thus measuring the focusing 
state of an image formed as an optical image . 

Reference numeral 4 6 denotes a photometry circuit 
used to execute the AE process. Light rays that have 
5 entered the lens 310 are guided to the photometry 

circuit 46 via the stop 312, the lens mounts 306 and 
106, the mirror 130 and a mirror 132, and a photometry 
lens (not shown) , thus measuring the exposure state of 
an image formed as an optical image. 

10 The photometry circuit 4 6 also has an EF process 

function in cooperation with an electronic flash 48. 

Reference numeral 48 denotes an electronic flash 
which has a function of projecting AF assist light, and 
a flash light control function. 

15 Note that the system control circuit 50 can 

perform exposure control and AF control using a video 
TTL scheme for the shutter control circuit 40, stop 
control circuit 340, and distance measurement control 
circuit 342 on the basis of the computation result of 

20 image data output from the image sensing element 14 by 
the image processing circuit 20. 

Furthermore, AF control may be done using both 
the measurement result of the distance measurement 
circuit 42 and the computation result of image data 

25 output from the image sensing element 14 by the image 
processing circuit 20. 



Moreover, exposure control may be done using both 
the measurement result of the photometry circuit 46 and 
the computation result of image data output from the 
image sensing element 14 by the image processing 
5 circuit 20. 

Reference numeral 50 denotes a system control 
circuit for controlling the overall image processing 
apparatus 100; and 52, a memory for storing constants, 
variables, programs, and the like required for 

10 operating the system control circuit 50. 

Reference numeral 54 denotes an indication unit 
which comprises a liquid crystal display device, 
loudspeaker, and the like, and indicates the operation 
state, messages, and the like using characters, images, 

15 sound, and the like in accordance with execution of 
programs in the system control circuit 50. The 
indication unit 54 is set at one or a plurality of 
easy-to-see positions around the console of the image 
processing apparatus 100, and is constructed by a 

20 combination of an LCD, LEDs, sound generation element, 
and the like. Some functions of the indication unit 54 
are set within an optical viewfinder 104. 

Of the indication contents of the indication unit 
54, those displayed on the LCD or the like include, 

25 e.g., a single/continuous shot indication, self timer 
indication, compression ratio indication, recording 
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pixel count indicating, recorded image count indication 
remaining recordable image count indication, shutter 
speed indication, aperture value indication, exposure 
correction indication, flash indication, red-eye 
5 suppression indication, macro image sensing indication, 
buzzer setup indication, remaining timepiece battery- 
capacity indication, remaining battery capacity 
indication, error indication, information indication 
using numerals of a plurality of digits, 

10 attachment /detachment indication of recording media 200 
and 210, attachment /detachment indication of a lens 
unit 300, communication I/F operation indication, 
date/time indication, connection indication with an 
external computer, and the like. 

15 Of the indication contents of the indication unit 

54, those displayed within the optical viewfinder 104 
include, e.g., in-focus indication, image sensing ready 
indication, camera shake alert indication, flash 
charging indication, flash charging completion 

20 indication, shutter speed indication, aperture value 
indication, exposure correction indication, recording 
medium write access indication, and the like. 

Furthermore, of the indication contents of the 
indication unit 54, those displayed using the LEDs and 

25 the like include, e.g., in-focus indication, image 

sensing ready indication, camera shake alert indication 



flash charging indication, flash charging completion 
indication, recording medium write access indication, 
macro image sensing setup notification, secondary 
battery charging state indication, and the like. 
5 Of the indication contents of the indication unit 

54, those indicated by lamps and the like include, e.g., 
a self timer notification lamp, and the like. The self 
timer notification lamp may be commonly used as AF 
assist light. 

10 Reference numeral 56 denotes an electrically 

erasable/programmable nonvolatile memory, which uses, 
e.g., an EE PROM or the like. 

Reference numerals 60, 62, 64, 66, 68, and 70 
denote operation units for inputting various operation 

15 instructions of a system control circuit 118. These 
operation units are constructed by one or a plurality 
of combinations of a switch, a dial, a touch panel, a 
pointer using line of sight detection, a voice 
recognition device, and the like. 

20 Examples of these operation units will be 

explained below. 

Reference numeral 60 denotes a mode dial switch 
which can selectively set one of various function image 
sensing modes: an automatic image sensing mode, 

25 programmed image sensing mode, shutter speed priority 
image sensing mode, aperture priority image sensing 



mode, manual image sensing mode, focal depth priority 
(depth) image sensing mode, portrait image sensing mode, 
landscape image sensing mode, macro image sensing mode, 
sport image sensing mode, night scene image sensing 
5 mode, panorama image sensing mode, and the like. 

Reference numeral 62 denotes a shutter switch SW1, 
which is turned on in the middle (half stroke position) 
of operation of a shutter button (not shown) , and 
instructs start of the AF (auto-focus) process, AE 

10 (auto-exposure) process, AWB (auto white balance) 

process, EF (flash light control) process, and the like. 

Reference numeral 64 denotes a shutter switch SW2, 
which is turned on upon completion of operation (at the 
full stroke position) of the shutter button (not shown) , 

15 and instructs start of a series of processes including 
an exposure process for writing a signal read out from 
the image sensing element 14 as image data in the 
memory 30 via the A/D converter 16 and memory control 
circuit 22, a development process using computation 

20 results in the image processing circuit 20 and memory 

control circuit 22, and a recording process for reading 
out image data from the memory 30, compressing the 
readout data by the compression/expansion circuit 32, 
and writing the compressed image data in the recording 

25 medium 2 00 or 210. 



Reference numeral 66 denotes an AEB (auto 
exposure bracketing) setup switch which is used to set 
an AEB image sensing mode that automatically brackets 
three frames with a standard exposure value, 
5 overexposure value, and underexposure value during 
single image sensing operation, and also to set a 
correction step value in this mode. 

Note that the correction step value can be set 
more smoothly if plus and minus buttons or a rotary 
10 dial switch is used. 

Reference numeral 68 denotes a single/continuous 
shot switch, which can set one of a single shot mode 
for sensing one frame of image upon depression of the 
shutter switch SW2, and then setting a standby state, 
15 and a continuous shot mode for successively sensing 
images while the shutter switch SW2 is held down. 

Reference numeral 70 denotes a console including 
various buttons, touch panel, and the like, which 
include a menu button, set button, macro button, 
20 multi-frame playback new page button, flash setup 

button, single shot/continuous shot/self timer switch 
button, menu move + (plus) button, menu move - (minus) 
button, playback image move + (plus) button, playback 
image move - (minus) button, sensed image quality 
25 select button, exposure correct button, date/time setup 
button, select/change button that can select and change 



various functions upon executing image sensing and 
playback in, e.g., the panoramic mode, 
determine/execute button which can determine and 
execute various functions upon executing image sensing 
5 and playback in, e.g., the panoramic mode, image 
display ON/OFF switch for turning on/off the image 
display unit 28, quick review ON/OFF setup switch for 
setting a quick review function of automatically 
playing back sensed image data immediately after image 

10 sensing, compression mode switch for selecting a 

compression ratio of JPEG compression, and selecting a 
CCDRAW mode for directly converting a signal output 
from the image sensing element into digital data, and 
recording the digital data in a recording medium, 

15 playback mode switch for setting various function modes 
such as a playback mode, multi-frame playback/delete 
mode, PC connect mode, and the like, AF mode setup 
switch which can set a one-shot AE mode for starting 
auto-focusing upon depression of the shutter switch SW1, 

20 and maintaining an in-focus state once it is attained, 
and a servo AF mode for continuously executing 
auto-focusing while the shutter switch SW1 is held down, 
playback switch which instructs to start playback 
operation for reading out an image sensed in a given 

25 image sensing mode from one of the memory 30 and the 



recording media 200 and 210, and displaying the readout 
image on the image display unit 28, and the like. 

In place of the plus and minus buttons, a rotary 
dial switch may be used to select numerical values and 
5 functions more smoothly. 

Reference numeral 72 denotes a power switch which 
can selectively set power-ON and power-OFF modes of the 
image processing apparatus 100. Also, the power switch 
72 can also selectively set power-ON and power-OFF 

10 modes of various accessories such as the lens unit 300, 
external electronic flash, recording media 200 and 210, 
and the like, which are connected to the image 
processing apparatus 100. 

Reference numeral 8 0 denotes a power supply 

15 control circuit which is comprised of a battery 

detection circuit, a DC-DC converter, a switch circuit 
for switching a block to be energized, and the like. 
The power supply control circuit 80 detects the 
presence/absence, type, and remaining battery amount of 

20 the battery attached, controls the DC-DC converter on 
the basis of such detection results and an instruction 
from the system control circuit 50, and supplies a 
required voltage to the respective units including the 
recording media for a required period of time. 

25 Reference numerals 82 and 8 4 denote connectors; 

and 8 6, a power supply circuit. The power supply 
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circuit 86 includes a primary battery such as an alkali 
battery, lithium battery, or the like, a secondary 
battery such as an NiCd battery, NiMH battery, Li 
battery, or the like, an AC adapter, and the like. 
5 Reference numerals 90 and 94 denote interfaces 

with recording media such as a memory card, hard disk, 
and the like; 92 and 96, connectors for connecting 
recording media such as a memory card, hard disk, and 
the like; and 98, a recording medium 

10 attachment/detachment detection circuit for detecting 
whether or not the recording medium 200 or 210 is 
attached to the connector 92 and/or the connector 96. 

Note that this embodiment has two sets of 
interfaces and connectors that receive the recording 

15 media. However, the number of sets of interfaces and 
connectors that receive the recording media is not 
particularly limited. Also, combinations of interfaces 
and connectors of different standards may be used. 

As the interface and connector, those complying 

20 with the standards of a PCMCIA card, CF (compact flash) 
card, and the like may be used. 

Furthermore, when the interfaces 90 and 94, and 
connectors 92 and 96 use those complying with the 
standards of a PCMCIA card, CF (compact flash) card, 

25 and the like, and various communication cards such as a 
LAN card, modem card, USB card, IEEE1394 card, P1284 



card, SCSI card, PHS, and the like are connected 
thereto, image data and associated management 
information can be transferred between the image 
processing apparatus and an external computer or its 
5 peripheral devices such as a printer and the like. 
Reference numeral 104 denotes an optical 
viewfinder which can guide light rays that have entered 
the lens 310 via the stop 312, lens mounts 306 and 106, 
and mirrors 130 and 132 by the single-lens reflex 

10 system, and can form and display them as an optical 
image. In this manner, without using the electronic 
viewfinder function implemented by the image display 
unit 28, image sensing can be done using the optical 
viewfinder 104 alone. In the optical viewfinder 104, 

15 some functions of the indication unit 54, e.g., an 
in-focus indication, camera shake alert indication, 
flash charging indication, shutter speed indication, 
aperture value indication, exposure correction 
indication, and the like are provided. 

20 Reference numeral 110 denotes a communication 

circuit having various communication functions such as 
RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, radio 
communication, and the like. 

Reference numeral 112 denotes a connector or 

25 antenna, which serves as a connector when the image 

processing apparatus 100 is connected to another device 



using the communication means 139, or serves as an 

antenna in case of radio communications. 

Reference numeral 120 denotes an interface for 

connecting the image processing apparatus 100 to the 
5 lens unit 300 in the lens mount 106; 122, a connector 

for electrically connecting the image processing 

apparatus 100 to the lens unit 300; and 124, a lens 

attachment /detachment detection circuit for detecting 

whether or not the lens unit 300 is attached to the 
10 lens mount 106 and/or the connector 122. 

The connector 122 also has a function of 

exchanging control signals, status signals, data 

signals, and the like between the image processing 

apparatus 100 and lens unit 300, and supplying currents 
15 of various voltages. The connector 122 may communicate 

not only electrical signals but also optical signals, 

audio signals, and the like. 

Reference numerals 130 and 132 denote mirrors 

which can guide light rays that have entered the lens 
20 310 to the optical viewfinder 104 by the single-lens 

reflex system. Note that the mirror 132 may be either 

a quick return mirror or half mirror. 

Reference numeral 200 denotes a recording medium 

such as a memory card, hard disk, or the like. The 
25 recording medium 200 comprises a recording unit 202 

comprised of a semiconductor memory, magnetic disk, or 



the like, an interface 204 with the image processing 
apparatus 100, and a connector 206 for connecting the 
image processing apparatus 100. 

Reference numeral 210 denotes a recording medium 
5 such as a memory card, hard disk, or the like. The 
recording medium 210 comprises a recording unit 212 
comprised of a semiconductor memory, magnetic disk, or 
the like, an interface 214 with the image processing 
apparatus 100, and a connector 216 for connecting the 

10 image processing apparatus 100. 

Reference numeral 300 denotes an exchangeable 
lens type lens unit. 

Reference numeral 306 denotes a lens mount for 
mechanically coupling the lens unit 300 to the image 

15 processing apparatus 100. The lens mount 306 includes 
various functions of electrically connecting the lens 
unit 300 and image processing apparatus 100. 

Reference numeral 310 denotes a photographing 
lens; and 312, a stop. 

20 Reference numeral 320 denotes an interface for 

connecting the lens unit 300 to the image processing 
apparatus 100 in the lens mount 306; and 322, a 
connector for electrically connecting the lens unit 300 
to the image processing apparatus 100. The connector 

25 322 also has a function of exchanging control signals, 
status signals, data signals, and the like between the 



image processing apparatus 100 and lens unit 300, and 
receiving or supplying currents of various voltages. 
Note that the connector 322 may communicate not only 
electrical signals but also optical signals, audio 
5 signals, and the like. 

Reference numeral 340 denotes a stop control 
circuit for controlling the stop 312 on the basis of 
photometry information from the photometry circuit 4 6 
in cooperation with the shutter control circuit 40 that 
10 controls the shutter 12. 

Reference numeral 342 denotes a distance 
measurement control circuit for controlling focusing of 
the photographing lens 310; and 344, a zoom control 
circuit for controlling zooming of the photographing 
15 lens 310. 

Reference numeral 350 denotes a lens system 
control circuit for controlling the overall lens unit 
300. The lens system control circuit 350 has a memory 
function of storing constants, variables, programs, and 
20 the like required for operating the lens unit 300, and 
a nonvolatile memory function of holding identification 
information such as a number or the like unique to the 
lens unit 300, management information, function 
information such as a full-aperture value, minimum 
25 aperture value, focal length, and the like,, current and 
past setup values, and the like. 
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The operation of the image processing apparatus 
with the above arrangement will be explained below with 
reference to Figs. 2 to 9 . 

Figs. 2 to 6 are flow charts of the main routine 
5 showing the control sequence of the image processing 
apparatus 100. 

Referring to Fig. 2, upon power ON after battery 
exchange or the like, the system control circuit 50 
initializes flags, control variables, and the like and 
10 also performs required predetermined initial setups in 
the respective units of the image processing apparatus 
100 (S101) . 

The system control circuit 50 checks the setup 
position of the power switch 72 (S102) . If the power 

15 switch 72 is set at the power-OFF position, the system 
control circuit 50 executes a predetermined end process 
{S103) . More specifically, the system control circuit 
50 changes the indications of the respective indication 
units to an end state, records required parameters and 

20 setup values including flags, control variables, and 
the like, and the currently selected mode in the 
nonvolatile memory 56, cuts off unnecessary power 
supply to the respective units of the image processing 
apparatus 100 including the image display unit 28 by 

25 the power supply control circuit 80, and so forth. 
After that, the flow returns to step S102. 



If the power switch 72 is set at the power-ON 
position (S102), the system control circuit 50 checks 
using the power supply control circuit 80 if the 
remaining capacity and operation state of the power 
5 supply 8 6 comprising batteries and the like pose any 
problem in the operation of the image processing 
apparatus 100 (S104) . If any problem is found, a 
predetermined alert indication is made by means of an 
image or voice using the indication unit 54 (S105) , and 

10 the flow then returns to step S102. 

If no problem is found in the power supply 86 
(S104), the system control circuit 50 checks the setup 
position of the mode dial switch 60 (S106) . If the 
mode dial switch 60 is set at one of the image sensing 

15 mode positions, the flow advances to step S108. 

If the mode dial switch 60 is set at any of other 
mode positions (S106) , the system control circuit 50 
executes a process corresponding to the selected mode 
(S107), and the flow returns to step S102 upon 

20 completion of the process. 

The system control circuit 50 checks if the 
recording medium 200 or 210 is attached, acquires 
management information of image data recorded on the 
recording medium 200 or 210, and then checks if the 

25 operation state of the recording medium 200 or 210 
poses any problem in the operation of the image 



processing apparatus 100, in particular, 
recording/playback of image data to/from the recording 
medium 200 or 210 (S108) . If any problem is found, a 
predetermined alert indication is made by means of an 
5 image or voice using the indication unit 54 (S105) , and 
the flow then returns to step S102. 

If no problem is found, the flow advances to step 
S109, and the system control circuit 50 checks the 
setup state of the single/continuous shot switch 68 

10 that sets the single/continuous shot mode (S109) . If 

the single shot mode is selected, the circuit 50 sets a 
single/continuous shot flag to indicate the single shot 
mode (S110) ; if the continuous shot mode is selected, 
the circuit 50 sets the single/continuous shot flag to 

15 indicate the continuous shot mode (Sill) . Upon 

completion of flag setups, the flow advances to step 
S112. 

With the single/continuous shot switch 68, the 
one-shot mode for sensing one frame of image upon 
20 depression of the shutter switch SW2, and then setting 
a standby state, and the continuous shot mode for 
successively sensing images while the shutter switch 
SW2 is held down can be arbitrarily selectively set. 

Note that the state of the single/continuous shot 
25 flag is stored in the internal memory of the system 
control circuit 50 or the memory 52. 
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Referring to Fig. 3, the system control circuit 
50 checks the setup state that pertains to the AE mode 
by the mode dial switch 60 (S112) . If one of the 
shutter speed (Tv value) priority image sensing modes 
5 is selected, the system control circuit 50 sets an AE 
mode flag to indicate a Tv priority mode (S113) ; if one 
of the aperture (Av value) priority image sensing modes 
is selected, the circuit 50 sets the AE mode flag to 
indicate an Av priority mode (S114) . Upon completion 

10 of flag setups, the flow advances to step S115. 

The shutter speed (Tv value) priority image 
sensing modes include the sport image sensing mode and 
the like in addition to the shutter speed priority 
image sensing mode. 

15 Also, the aperture (Av value) priority image 

sensing modes include the portrait image sensing mode, 
landscape image sensing mode, focal depth priority 
(depth) image sensing mode, and the like in addition to 
the aperture priority image sensing mode. 

20 Note that the state of the AE mode flag is stored 

in the internal memory of the system control circuit 50 
or the memory 52. 

The system control circuit 50 checks the setup 
state of the AEB setup switch 66 that sets the AEB 

25 image sensing mode that performs automatic bracketing 
(S115) . If the AEB image sensing mode is set, the 



system control circuit 50 sets an AEB flag (S116) , and 
also sets a correction step value (S117) . The flow 
then advances to step S119. 

On the other hand, if the AEB image sensing mode 
5 is not set, the system control circuit 50 resets the 
AEB flag (S118), and the flow advances to step S119. 

The correction step value can set an exposure 
step difference upon exposing three frames with a 
standard exposure value, overexposure value, and 

10 underexposure value upon automatic bracketing. The 
correction step value normally uses values such as a 
1/3 step, 1/2 step, 1 step, and the like. 

Note that the state of the AEB flag and AEB 
correction step value are stored in the internal memory 

15 of the system control circuit 50 or the memory 52. 

If the AEB setup switch 66 is reset, the AEB 
correction step value is set at zero step, or 
predetermined setups for canceling image sensing during 
AEB image sensing are made, the system control circuit 

20 50 determines that the setups for ending the AEB image 
sensing mode in the middle of the process are made 
(S119) , and resets an AEB execution flag stored in its 
internal memory or the memory 52 (S121) . The flow then 
returns to step S102. 

25 If the setups for ending the AEB image sensing 

mode in the middle of the process are not made (S119) , 



the system control circuit 50 indicates various setup 
states of the image display apparatus 100 by means of 
an image or voice using the indication unit 54 (S120) . 
When the image display of the image display unit 28 is 
5 ON, various setup states of the image display apparatus 
100 are indicated by means of an image also using the 
image display unit 28. 

Referring to Fig. 4, if the shutter switch SW1 is 
OFF (S131) , the flow returns to step S102. 

10 If the shutter switch SW1 is ON (S131), the 

system control circuit 50 checks the state of the AEB 
execution flag stored in its internal memory or the 
memory 52 (S132) . If the AEB execution flag is set, 
the system control circuit 50 determines that AEB image 

15 sensing has already been started, and the flow advances 
to step S141. 

If the AEB execution flag is reset (S132) , the 
system control circuit 50 initializes a variable N to N 
= 0 (S133) , and the flow then advances to step S134. 

20 The system control circuit 50 executes a distance 

measurement /photometry process (S134) . More 
specifically, the system control circuit 50 executes a 
distance measurement process for bringing the 
photographing lens 310 into focus on an object, and 

25 also executes a photometry process to determine the 
aperture value and shutter speed. The flow then 



advances to step S135. In the photometry process, the 
electronic flash is set if necessary. 

The distance measurement /photometry process 
(S134) will be described in detail later with reference 
5 to Fig. 7. 

The system control circuit 50 checks the state of 
the AE mode flag stored in its internal memory or the 
memory 52 (S135) , and determines a reference shutter 
speed used to compute the charge accumulation time of 
10 the image sensing element 14 to be set in step S138 in 
correspondence with the state of the AE mode flag. 

If the AE mode flag is set to indicate the Av 
priority mode (S135) , the system control circuit 50 
determines a value obtained by adding the correction 
15 step value set in step S117 to the shutter speed (Tv 
value) obtained in step S134 to be the reference 
shutter speed based on which the charge accumulation 
time is computed (S136) . The determined reference 
shutter speed corresponds to a shutter speed (slow 
20 shutter speed) used to sense an image with an 
overexposure value in AEB image sensing. 

In this manner, when the reference shutter speed 
is set at a shutter speed used to sense an image with 
an overexposure value, and the dark capture process is 
25 done according to this speed, the shutter speed need 

not be changed for each of three frames upon executing 



AEB image sensing in the aperture (Av value) priority 
image sensing mode. That is, even when the dark 
capture process is done for a longer period of time 
than the regular time, no problem is posed in a 
5 correction process based on that data, thus allowing 
such setups. Hence, the reference shutter speed may be 
set to be a value obtained by adding a value slightly 
larger than the correction step value set in step S117 
to the shutter speed (Tv value) obtained by the 

10 distance measurement /photometry process (S134) . 

On the other hand, if the AE mode flag is set to 
indicate the Tv priority mode (S135), the system 
control circuit 50 determines the shutter speed (Tv 
value) obtained by the distance measurement /photometry 

15 process (S134) to be the reference shutter speed (S137) . 

In this manner, when the AE mode flag is set to 
indicate the Tv priority mode (shutter speed priority 
mode) , since image sensing must be done for three 
frames at equal shutter speed upon AEB image sensing in 

20 the shutter speed priority mode, the shutter speed 

obtained by the distance measurement /photometry process 
(S134) is selected as the reference shutter speed used 
to compute the charge accumulation time. 

The system control circuit 50 sets the charge 

25 accumulation time of the image sensing element 14 in 

accordance with the reference shutter speed determined 



in step S136 or S137 (S138), and the flow advances to 
step S139. 

Note that charge accumulation is made using the 
charge accumulation time set in this step in an image 
5 sensing process (S164) and a dark capture process (S140 
or S167), which will be described later. 

The system control circuit 50 checks the state of 
the single/continuous shot flag stored in its internal 
memory or the memory 52 (S139) . If the single shot 
10 mode is set, the flow jumps to step S141. 

In this manner, if it is determined in step S139 
that the single shot mode is selected, the flow jumps 
to step S141 without executing a dark capture process 
(S140) . With this control, a release time lag upon 
15 depression of the shutter switch SW2 in step S141 can 
be reduced. 

If the continuous shot mode is selected (S139), 
the system control circuit 50 executes a dark capture 
process (S140) . That is, the system control circuit 50 

20 accumulates noise components such as dark current and 
the like of the image sensing element 14 for the same 
period of time as that required for actual image 
sensing, while closing the shutter 12, and reads out 
the accumulated noise image signal. After that, the 

25 flow advances to step S141. 
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The dark capture data captured in this dark 
capture process is used in a correction computation 
process later, and the correction process is made for 
image data that suffers image quality deterioration 
5 such as pixel omission or the like caused by dark 

current noise produced by the image sensing element 14 
and scratches unique to the image sensing element 14. 

The dark capture process (S140) will be described 
in detail later with reference to Fig. 9. 

10 In this manner, if it is determined in step S139 

that the continuous shot mode is set, the dark capture 
process (S140) is executed prior to continuous shot 
image sensing, thus setting nearly constant continuous 
shot frame intervals upon executing continuous shot 

15 image sensing by pressing the shutter switch SW2 in 
step S141. 

The system control circuit 50 checks in step S141 
if the shutter switch SW2 is ON. If the shutter switch 
SW2 is OFF, the system control circuit 50 repeats the 
20 current process until the shutter switch SW1 is turned 
off (S142) . 

If the shutter switch SW1 is turned off (S142), 
the flow returns to step S102. 

If the shutter switch SW2 is ON (S141), the 
25 system control circuit 50 checks if an area that can 
store sensed image data is available on an image 



storage buffer area on the memory 30 (S161) , as shown 
in Fig. 5. If no area that can store sensed image data 
is available on the image storage buffer area on the 
memory 30, a predetermined alert indication is made by 
5 means of an image or voice using the indication unit 54 
(S162), and the flow then returns to step S102. 

For example, the user experiences such state 
immediately after he or she has executed continuous 
shot image sensing corresponding to a maximum number of 

10 images that can be stored in the image storage buffer 
area. In this state, the first image to be read out 
from the memory 30 and written in the recording medium 
200 or 210 is not recorded on the recording medium 200 
or 210 yet, and a free area even for one image cannot 

15 be assured on the image storage buffer area on the 
memory 30. 

When sensed image data is stored in the image 
storage buffer area on the memory 30 after it is 
compressed, it is checked in step S161 if an area that 

20 can store sensed image data is available on the image 
storage buffer area on the memory 30, in consideration 
of the fact that the compressed image data size varies 
depending on the setups of the compression mode. 

If an area that can store sensed image data is 

25 available on the image storage buffer area on the 

memory 30 (S161), the system control circuit 50 checks 
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the state of the AEB flag stored in its internal memory 
or the memory 52 (S181) . If the AEB flag is set, the 
flow advances to step S163; otherwise, the flow jumps 
to step S164. 

5 In step S163, the system control circuit 50 sets 

the AEB execution flag. Note that the state of the AEB 
execution flag is stored in the internal memory of the 
system control circuit 50 or the memory 52. 

The system control circuit 50 then checks the 
10 value of the variable N. If N = 0, the system control 
circuit 50 sets AEB correction photometry data for 
underexposure, and stores it in its internal memory or 
the memory 52 (S182) . 

If N = 1, the system control circuit 50 sets AEB 
15 correction photometry data for standard exposure and 
stores it in its internal memory or the memory 52 
(S183) . 

If N = 2, the system control circuit 50 sets AEB 
correction photometry data for overexposure and stores 
20 it in its internal memory or the memory 52 (S184). 

The system control circuit 50 then executes an 
image sensing process (S164). More specifically, the 
system control circuit 50 reads out a sensed image 
signal, which has been accumulated for a predetermined 
25 period of time upon image sensing, from the image 

sensing element 14, and writes the sensed image data on 
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the predetermined area of the memory 30 via the A/D 
converter 16, image processing circuit 20, and memory 
control circuit 22, or from the A/D converter 16 
directly via the memory control circuit 22. 
5 The image sensing process (S164) will be 

explained in detail later using Fig. 8. 

Upon completion of the image sensing process 
(S164) , the system control circuit 50 checks the state 
of the AEB flag stored in its internal memory or the 
10 memory 52 (S185) . If the AEB flag is set, the flow 
advances to step S165; otherwise, the flow jumps to 
step S166. 

In step S165, the system control circuit 50 
updates the variable N to N = N + 1, and the flow then 

15 advances to step S166. 

The system control circuit 50 checks the state of 
the single/continuous shot flag stored in its internal 
memory or the memory 52 (S166) . If the continuous shot 
mode is set, the flow advances to step S168. 

20 If it is determined in step S166 that the 

continuous shot mode is set, since the dark capture 
process has already been done in step S140 prior to 
continuous shot image sensing, a development process 
(S168) is executed without executing another dark 

25 capture process in step S167. In this manner, nearly 
constant continuous shot frame intervals can be set. 



If the single shot mode is set (S166) , the system 
control circuit 50 executes the dark capture process 
for accumulating noise components such as dark current 
and the like of the image sensing element 14 for the 
5 same period of time as that reguired for actual image 
sensing, while closing the shutter 12, and reading out 
the accumulated noise image signal (S167) . The flow 
then advances to step S168. 

By making a correction computation process later 
10 using dark image data captured by the dark capture 
process, image data that suffers image quality 
deterioration such as pixel omission or the like caused 
by dark current noise produced by the image sensing 
element 14 and scratches unique to the image sensing 
15 element 14 can be corrected. 

The dark capture process (S167) will be described 
in detail later using Fig. 9. 

In this manner, if it is determined in step S166 
that the single shot mode is set, the dark capture 
20 process (S167) is executed after the image sensing 

process (S164) , thus reducing the shutter release time 
lag (the time lag from when SW2 is pressed until image 
sensing starts actually) upon depression of the shutter 
switch SW2 in step S141. 
25 In step S168 in Fig. 6, the system control 

circuit 50 reads out some of image data written in the 



predetermined area of the memory 30 via the memory 
control circuit 22, executes a WB {white balance) 
integral computation process and OB (optical black) 
integral computation process required to execute 
development processes, and stores the computation 
results in its internal memory or the memory 52. 

The system control circuit 50 then reads out 
sensed image data written in the predetermined area of 
the memory 30 using the memory control circuit 22 and 
the image processing circuit 20 as needed, and executes 
various development processes including AWB (auto white 
balance) process, gamma conversion, color conversion, 
and the like using the computation results stored in 
its internal memory or the memory 52. 

Furthermore, in the development processes, the 
system control circuit 50 also executes a dark 
correction computation process for canceling dark 
current noise and the like of the image sensing element 
14 by executing a subtraction process using the dark 
image data captured in the dark capture process. 

The system control circuit 50 reads out image 
data written in the predetermined area of the memory 30, 
makes the compression/expansion circuit 32 execute an 
image compression process in accordance with the 
selected mode (S169) , and writes the obtained image 



data in a free image space of the image storage buffer 
area on the memory 30. 

Upon executing a series of processes, the system 
control circuit 50 executes a recording start process 
5 (S170) . That is, the system control circuit 50 reads 
out image data stored in the image storage buffer area 
of the memory 30, and begins to write the readout image 
data in the recording medium 200 or 210 such as a 
memory card, compact flash card, or the like. 

10 This recording start process is executed for new 

image data, which has been sensed and undergone a 
series of processes, every time that image data is 
written in a free image space of the image storage 
buffer area on the memory 30. 

15 While a write of image data in the recording 

medium 200 or 210 is underway, a recording medium write 
access indication (e.g., flashing an LED of the 
indication unit 54) is made to clearly indicate that 
write access. 

20 The system control circuit 50 checks the state of 

the AEB flag stored in its internal memory or the 
memory 52 (S186) . If the AEB flag is set, the flow 
advances to step S171; otherwise, the flow jumps to 
step S173. 

25 The system control circuit 50 checks in step S171 

if the variable N has reached N = 3. If the variable N 
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has reached N = 3 , the system control circuit 50 resets 
the AEB execution flag (S172), and the flow returns to 
step S102. 

If the variable N has not reached "n = 3 yet 
5 (S171), the system control circuit 50 checks the state 
of the single/continuous shot flag stored in its 
internal memory or the memory 52 (S173) . If the 
continuous shot mode is set, the flow returns to step 
S141 to proceed with image sensing, thus making the 
10 next image sensing. 

If the single shot mode is set, the system 
control circuit 50 checks if the shutter switch SW1 is 
ON (S174) . 

If the shutter switch SW1 is ON (S174), the 
15 system control circuit 50 repeats the current process 
until the shutter switch SW1 is turned off. If the 
shutter switch SW1 is OFF (S174), the flow returns to 
step S102. 

Fig. 7 is a flow chart showing the detailed 
20 sequence of the distance measurement/photometry process 
in step S134 in Fig. 4. In the distance 
measurement /photometry process, the system control 
circuit 50 and the stop control circuit 340 or distance 
measurement control circuit 342 exchange various 
25 signals via the interface 120, connectors 122 and 322, 
interface 320, and lens control circuit 350. 



The system control circuit 50 starts an AF 
(auto-focus) process using the image sensing element 14, 
distance measurement circuit 42, and distance 
measurement control circuit 342 (S201) . 
5 The system control circuit 50 executes AF control 

for checking the focusing state of an image formed as 
an optical image by guiding light rays, which have 
entered the lens 310, to the distance measurement 
circuit 42 via the stop 312, lens mounts 306 and 106, 

10 mirror 130, and distance measurement sub-mirror (not 
shown) , and detecting the focusing state using the 
distance measurement circuit 42 while driving the lens 
310 using the distance measurement control circuit 342 
(S202), until the distance measurement (AF) result 

15 indicates an in-focus (S203) . 

If the distance measurement (AF) result indicates 
an in-focus (S203) , the system control circuit 50 
determines an in-focus distance measurement point from 
those in an image sensing screen (S204), and stores 

20 distance measurement data and/or setup parameters in 

its internal memory or the memory 52 together with the 
determined distance measurement point data. The flow 
then advances to step S205. 

Subsequently, the system control circuit 50 

25 starts an AE (auto-exposure) process using the 
photometry circuit 46 (S205) . 



The system control circuit 50 executes a 
photometry process using the shutter control circuit 40 

(5206) for measuring the exposure state of an image 
formed as an optical image by guiding light rays, which 

5 have entered the lens 310, to the photometry circuit 4 6 
via the stop 312, lens mounts 306 and 106, mirrors 130 
and 132, and photometry lens (not shown), until it is 
determined that the exposure (AE) value is appropriate 

(5207) . 

10 If it is determined that the exposure (AE) value 

is appropriate (S207), the system control circuit 50 
stores photometry data and/or setup parameters in its 
internal memory or the memory 52, and the flow advances 
to step S208. 

15 Note that the system control circuit 50 

determines the aperture value (Av value) and shutter 
speed (Tv value) in accordance with the exposure (AE) 
result detected by the photometry process (S206) and 
the image sensing mode selected by the mode dial switch 

20 60. 

The system control circuit 50 determines the 
charge accumulation time of the image sensing element 
14 in accordance with the determined shutter speed (Tv 
value) , and executes the image sensing process and dark 
25 capture process using an egual charge accumulation time. 



The system control circuit 50 checks based on the 
measurement data obtained by the photometry process 
(S206) if flash emission is required (S208) . If flash 
emission is required, the circuit 50 sets a flash flag, 
5 and charges the electronic flash 48 {S209) until 

charging of the electronic flash 48 is completed (S210) . 

Upon completion of charging of the electronic 
flash 48 (S210) , the distance measurement/photometry 
process routine (S134) ends. 
1° Fig. 8 is a flow chart showing the detailed 

sequence of the image sensing process in step S164 in 
Fig. 5. 

In this image sensing process, the system control 
circuit 50 and the stop control circuit 340 or distance 

15 measurement control circuit 342 exchange various 

signals via the interface 120, connectors 122 and 322, 
interface 320, and lens system control circuit 350. 

The system control circuit 50 moves the mirror 
130 to a mirror up position using a mirror driving 

20 circuit (not shown) (S301), and drives the stop 312 to 
a predetermined aperture value by the stop control 
circuit 34 0 in accordance with photometry data stored 
in its internal memory or the memory 52 if the AEB flag 
is not set, or the AEB correction photometry data if 

25 the AEB flag is set (S302) . 
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The system control circuit 50 clears charge on 
the image sensing element 14 (S303) , then starts charge 
accumulation of the image sensing element 14 (S304), 
opens the shutter 12 by the shutter control circuit 40 
5 (S305) , and starts exposure of the image sensing 
element 14 (S306) . 

The system control circuit 50 checks based on the 
flash flag if the electronic flash 48 is required 
(S307), and if the electronic flash 48 is required, the 
10 circuit 50 controls the electronic flash 48 to emit 
light (S308) . 

The system control circuit 50 waits for the end 
of exposure of the image sensing element 14 in 
accordance with the photometry data if the AEB flag is 
15 not set, or the AEB correction photometry data if the 
AEB flag is set (S309), closes the shutter 12 by the 
shutter control circuit 40 (S310) , and ends exposure of 
the image sensing element 14. 

The system control circuit 50 drives the stop 312 
20 to a full-open aperture value by the stop control 
circuit 340 (S311), and moves the mirror 130 to a 
mirror down position by the mirror driving circuit (not 
shown) (S312) . 

If the determined charge accumulation time has 
25 elapsed (S313), the system control circuit 50 reads out 
a charge signal from the image sensing element 14 upon 



completion of charge accumulation of the image sensing 
element 14 (S314), and writes sensed image data in a 
predetermined area of the memory 30 via the A/D 
converter 16, image processing circuit 20, and memory 
5 control circuit 22, or from the A/D converter 16 
directly via the memory control circuit 22 (S315) . 
Upon completion of a series of processes, the image 
sensing process routine (S164) ends. 

Fig. 9 is a flow chart showing the detailed 
10 seguence of the dark capture process in step S140 in 
Fig. 4 and in step S167 in Fig. 5. 

The system control circuit 50 clears charge on 
the image sensing element 14 (S401) , and then starts 
charge accumulation of the image sensing element 14 
15 while the shutter 12 is closed (S402) . 

If the predetermined charge accumulation time has 
elapsed (S403) , the system control circuit 50 ends 
charge accumulation of the image sensing element 14 
(S404), then reads out a charge signal from the image 
20 sensing element 14, and writes image data (dark image 
data) in a predetermined area of the memory 30 via the 
A/D converter 16, image processing circuit 20, and 
memory control circuit 22, or from the A/D converter 16 
directly via the memory control circuit 22 (S405) . 
25 By executing the development process using this 

dark capture data, sensed image data can be corrected 
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for image quality deterioration such as pixel omission 
or the like caused by dark current noise produced by 
the image sensing element 14 and scratches unique to 
the image sensing element 14. 
5 Note that this dark image data is held in the 

predetermined area of the memory 30 until a new 
distance measurement /photometry process is done or the 
power switch of the image processing apparatus 100 is 
turned off. 

10 This dark image data is used when the next image 

sensing process is done, and image data sensed by that 
process is read out to execute the development process. 

Or when image data sensed by the image sensing 
process and read out from the image sensing element 14 
15 has already been written in the memory 30, the 

development process is executed using dark image data. 

Upon completion of a series of processes, the 
dark capture process routine (S140 and S167) ends. 

Fig. 10 is a timing chart showing the flow of 
20 image sensing operation in the aforementioned 
embodiment . 

In the above embodiment, the exposure values upon 
AEB image sensing are set to sense images in the order 
of underexposure, standard exposure, and overexposure. 
25 However, the present invention is not limited to such 
specific order, and the exposure values may be set to 
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sense images in any other orders such as in the order 
of standard exposure, underexposure, and overexposure, 
of standard exposure, overexposure, and underexposure, 
of overexposure, standard exposure, and underexposure, 
5 and the like. 

In the above embodiment, in continuous shot AEB 
image sensing, immediately after the dark capture 
process is executed once, images with underexposure, 
standard exposure, and overexposure values are sensed. 

10 In single shot AEB image sensing, a total of three dark 
capture processes are executed every time each of 
images with underexposure, standard exposure, and 
overexposure values is sensed. Instead, in both 
continuous shot AEB image sensing and single shot AEB 

15 image sensing, the dark capture process may be executed 
once in correspondence with the longest charge 
accumulation time. 

In this case, in single shot AEB image sensing, 
the dark capture process is executed once in 

20 correspondence with the longest charge accumulation 
time in the first image sensing operation, and dark 
correction computations are made using the first dark 
capture data without executing any dark capture process 
in the second and third image sensing operations. With 

25 this control, the image sensing interval for the second 



- 47 - 



and subsequent frames upon single shot AEB image 
sensing can be shortened. 

In the above embodiment, the single or continuous 
shot mode is selected using the single/continuous shot 
5 switch 68. Alternatively, the single or continuous 
shot mode may be selected in correspondence with the 
operation mode selected by the mode dial switch 60. 

The AEB image sensing mode and AEB correction 
step value are simultaneously set by the AEB setup 
10 switch. Alternatively, the AEB image sensing mode and 
AEB correction step value may be independently set at 
quite different timings. 

In this case, independent operation switches may 
be used to independently set the AEB image sensing mode 
15 and AEB correction step value. 

In the above description, the AE mode is set in 
correspondence with the image sensing mode set by the 
mode dial switch 60. Alternatively, an AE mode setup 
switch may be added, and the AE mode may be set 
20 independently of the mode dial switch 60. 

In the above description, the number of frames to 
be sensed in AEB image sensing is three. However, the 
present invention is not limited to such specific 
number of frames, as long as a plurality of frames are 
25 sensed. 



Also, the correction step value in AEB image 
sensing need not be constant for the respective frames 
to be sensed. For example, the step value between the 
standard and overexposure values may be set at 1/2 step, 
5 and that between the standard exposure value and 
underexposure value may be set at 1/3 step. 

Such different correction step values may be set 
by the user of the image sensing apparatus 100. 

Furthermore, the image sensing apparatus 100 
10 itself may set different values for the overexposure 
and underexposure sides in accordance with grayscale 
characteristics unique to the image sensing element 14 
to improve the AEB image sensing effect. 

In the above embodiment, the charge accumulation 
15 time of the actual image sensing process is set to be 
equal to that of the dark capture process. However, 
different charge accumulation times may be set as long 
as sufficient data that can be used to correct dark 
current noise and the like can be obtained. 
20 During execution of the dark capture process in 

steps S140 and S167, since image sensing cannot be done, 
a visual or audio message indicating that the image 
processing apparatus 100 is busy may be made using the 
indication unit 54 and/or image display unit 28. 
25 In the above embodiment, image sensing is done by 

moving the mirror 130 between the mirror up and down 



positions. However, the mirror 130 may comprise a half 
mirror, and image sensing may be done without moving 
the mirror. 

The recording media 200 and 210 are not limited 
5 to memory cards such as PCMCIA cards, compact flash 
cards, or the like, hard disks, and the like, but may 
use micro DATs (digital audio tapes) , magnetooptical 
disks, optical disks such as CD-Rs, CD-WRs, or the like, 
phase change optical disks such as DVDs, and the like. 
10 Also, the recording media 200 and 210 may use 

hybrid media that integrate memory cards, hard disks, 
and the like. Furthermore, such hybrid media may 
include detachable media. 

In the above embodiment, the recording media 200 
15 and 210 are independent from the image processing 

apparatus 100 and are arbitrarily connectable. One or 
both the recording media 200 and 210 may be permanently 
connected to the image processing apparatus 100. 

An arbitrary number (one or a plurality) of image 
20 recording media 200 or 210 may be connectable to the 
image processing apparatus 100. 

In the above description, the recording media 200 
and 210 are attached to the image processing apparatus. 
However, one or a combination of a plurality of 
25 recording media may be used. 



Note that the present invention may be applied to 
either a system constituted by a plurality of devices, 
or an apparatus consisting of a single equipment. 

The objects of the present invention are also 
5 achieved by supplying a storage medium, which records a 
program code of a software program that can implement 
the functions of the above-mentioned embodiments to the 
system or apparatus, and reading out and executing the 
program code stored in the storage medium by a computer 
10 (or a CPU or MPU) of the system or apparatus. 

In this case, the program code itself read out 
from the storage medium implements the functions of the 
above-mentioned embodiments, and the storage medium 
which stores the program code constitutes the present 
15 invention. 

As the storage medium for supplying the program 
code, for example, a floppy disk, hard disk, optical 
disk, magnetooptical disk, CD-ROM, CD-R, magnetic tape, 
nonvolatile memory card, ROM, and the like may be used. 
20 The functions of the above-mentioned embodiments 

may be implemented not only by executing the readout 
program code by the computer but also by some or all of 
actual processing operations executed by an OS 
(operating system) running on the computer on the basis 
25 of an instruction of the program code. 



Furthermore, the functions of the above-mentioned 
embodiments may be implemented by some or all of actual 
processing operations executed by a CPU or the like 
arranged in a function extension board or a function 
5 extension unit, which is inserted in or connected to 
the computer, after the program code read out from the 
storage medium is written in a memory of the extension 
board or unit. 

The embodiments of the present invention have 
10 been described. However, the present invention is not 
limited to the arrangements of such embodiments, but 
may be applied to any other arrangements as long as 
they may achieve functions defined in the appended 
claims or functions of the arrangements of the 
15 embodiments. 

The software and hardware arrangements of the 
above embodiments can be appropriately replaced. 

The present invention may be achieved by 
combining the above embodiments or their technical 
20 components as needed. 

The present invention may be applied even when 
all or some of claims or the arrangements of the 
embodiments form a single apparatus, are combined with 
another apparatus, or are building components of the 
25 apparatus. 
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The present invention can be applied to various 
types of cameras such as an electronic still camera, 
video movie camera, a camera using a silver halide film, 
and the like, image sensing apparatuses other than 
5 cameras, devices applied to such cameras and image 

sensing apparatuses, and building components of those 
devices . 



WHAT IS CLAIMED IS: 

1. An apparatus comprising: 

(A) an image sensing device; 

(B) a signal processing device for performing a 
5 first image sensing operation for making said image 

sensing device perform an image sensing operation in an 
exposure state to obtain a sensed image signal, and a 
second image sensing operation for making said image 
sensing device perform an image sensing operation in 

10 accordance with said first image sensing operation in a 
non-exposure state to obtain a sensed image signal, and 
processing the sensed image signal obtained by the 
first image sensing operation by the sensed image 
signal obtained by the second image sensing operation, 

15 wherein said signal processing device changes method of 
the second image sensing operation to the first image 
sensing operation in response to the instruction of 
said instruction device; and 

(C) an instruction device for instructing to 
20 execute a predetermined plurality times of image 

sensing operations with different image sensing times 
of said first image sensing operation. 

2. The apparatus according to claim 1, wherein said 
signal processing device selects an image sensing time 
25 of the first image sensing operation of a predetermined 
one of the plurality times of image sensing operations 
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in response to the instruction of said instruction 
device, and makes the second image sensing operation 
for the plurality times of image sensing operations in 
accordance with the selected image sensing time. 

3. The apparatus according to claim 2, further 
comprising: 

a storage device for storing the sensed image 
signal of the second image sensing operation according 
to the selected image sensing time. 

4. The apparatus according to claim 3, wherein said 
signal processing device processes the sensed image 
signal obtained by the first image sensing operation on 
the basis of the sensed image signal stored in said 
storage device in image sensing operations other than 
the predetermined one of the plurality times of image 
sensing operations. 

5. The apparatus according to claim 1, wherein said 
signal processing device selects a longest image 
sensing time of the first image sensing operation of 
the plurality times of image sensing operations in 
response to the instruction of said instruction device, 
and makes the second image sensing operation for the 
plurality times of image sensing operations in 
accordance with the selected image sensing time. 

6. The apparatus according to claim 5, further 
comprising : 



a storage device for storing the sensed image 
signal of the second image sensing operation according 
to the selected image sensing time. 

7. The apparatus according to claim 6, wherein said 
5 signal processing device processes the sensed image 

signal obtained by the first image sensing operation on 
the basis of the sensed image signal stored in said 
storage device in image sensing operations other than 
the longest image sensing time of the first image 
10 sensing operation of the plurality times of image 
sensing operations. 

8. The apparatus according to claim 1, wherein said 
signal processing device designates an image sensing 
time of the first image sensing operation of a 

15 predetermined image sensing operation in response to 
the instruction of said instruction device, and makes 
the second image sensing operation for the plurality 
times of image sensing operations in accordance with 
the designated image sensing time. 

20 9. The apparatus according to claim 8, further 

comprising : 

a storage device for storing the sensed image 
signal of the second image sensing operation according 
to the designated image sensing time. 
25 10. The apparatus according to claim 9, wherein said 
signal processing device processes the sensed image 



signal obtained by the first image sensing operation on 
the basis of the sensed image signal stored in said 
storage device in image sensing operations other than 
the predetermined one of the plurality of image sensing 
5 operations. 

11. The apparatus according to claim 1, wherein said 
apparatus includes a camera. 

12. An apparatus comprising: 

(A) an image sensing device; 

10 (B) a signal processing device for performing a 

first image sensing operation for making said image 
sensing device perform an image sensing operation in an 
exposure state to obtain a sensed image signal, and a 
second image sensing operation for making said image 

15 sensing device perform an image sensing operation in a 
non-exposure state to obtain a sensed image signal, and 
processing the sensed image signal obtained by the 
first image sensing operation by the sensed image 
signal obtained by the second image sensing operation, 

20 wherein said signal processing device inhibiting the 

second image sensing operation from being made for each 
image sensing operation in response to the instruction 
of said instruction device; and 

(C) an instruction device for instructing to 

25 execute a predetermined plurality times of image 



sensing operations with different image sensing times 
of said first image sensing operation. 

13. The apparatus according to claim 12, wherein said 
signal processing device selects an image sensing time 

5 of the first image sensing operation of a predetermined 
one of the plurality times of image sensing operations 
in response to the instruction of said instruction 
device, and makes the second image sensing operation 
for the plurality times of image sensing operations in 
10 accordance with the selected image sensing time. 

14. The apparatus according to claim 13, further 
comprising : 

a storage device for storing the sensed image 
signal of the second image sensing operation according 
15 to the selected image sensing time. 

15. The apparatus according to claim 14, wherein said 
signal processing device processes the sensed image 
signal obtained by the first image sensing operation on 
the basis of the sensed image signal stored in said 

20 storage device in image sensing operations other than 
the predetermined one of the plurality times of image 
sensing operations. 

16. The apparatus according to claim 12, wherein said 
signal processing device selects a longest image 

25 sensing time of the first image sensing operation of 
the plurality times of image sensing operations in 



response to the instruction of said instruction device, 
and makes the second image sensing operation for the 
plurality times of image sensing operations in 
accordance with the selected image sensing time. 
5 17. The apparatus according to claim 16, further 
comprising : 

a storage device for storing the sensed image 
signal of the second image sensing operation according 
to the selected image sensing time. 

10 18. The apparatus according to claim 17, wherein said 
signal processing device processes the sensed image 
signal obtained by the first image sensing operation on 
the basis of the sensed image signal stored in said 
storage device in image sensing operations other than 

15 the longest image sensing time of the first image 

sensing operation predetermined one of the plurality 
times of image sensing operations. 

19. The apparatus according to claim 12, wherein said 
signal processing device designates an image sensing 

20 time of the first image sensing operation of a 

predetermined image sensing operation in response to 
the instruction of said instruction device, and makes 
the second image sensing operation for the plurality 
times of image sensing operation in accordance with the 

25 designated image sensing time. 



- 59 - 



20. The apparatus according to claim 19, further 
comprising: 

a storage device for storing the sensed image 
signal of the second image sensing operation according 
to the designated image sensing time. 

21. The apparatus according to claim 20, wherein said 
signal processing device processes the sensed image 
signal obtained by the first image sensing operation on 
the basis of the sensed image signal stored in said 
storage device in image sensing operations other than 
the predetermined one of the plurality of image sensing 
operations . 

22. The apparatus according to claim 12, wherein said 
apparatus includes a camera. 
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ABSTRACT OF THE DISCLOSURE 
An apparatus has an image sensing device, an 
instruction device for instructing to execute a 
plurality of image sensing operations with different 
image sensing times of the image sensing device, and a 
signal processing device for performing a first image 
sensing operation for making the image sensing device 
perform an image sensing operation in an exposure state 
to obtain a sensed image signal, and a second image 
sensing operation for making the image sensing device 
perform an image sensing operation in a non-exposure 
state to obtain a sensed image signal, and processing 
the sensed image signal obtained by the first image 
sensing operation by the sensed image signal obtained 
by the second image sensing operation. The signal 
processing device changes the second image sensing 
operation in response to the instruction of the 
instruction device. 
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To the inventor(s): 

The following are cited in or pertinent to the declaration attached to the accompanying application: 

Title 37. Code of Federal Regulation. § 1.56 

Duty to disclose information material to patentability. 

(a) A patent by its very nature is affect with a public interest The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, 
the Office is aware of and evaluates the teachings of all information material to patentability. Each 
individual associated with the filing and prosecution of a patent application has a duty of candor and 
good faith in dealing with the Office, which includes a duty to disclose to the Office all information 
known to that individual to be material to patentability as defined in this section. The duty to disclose 
information exists with respect to each pending claim until the claim is canceled or withdrawn from 
consideration, or the application becomes abandoned. Information material to the patentability of a 
claim that is canceled or withdrawn from consideration need not be submitted if the information is not 
material to the patentability of any claim remaining under consideration in the application. There is no 
duty to submit information which is not material to the patentability of any existing claim. The duty to 
disclose all information known to be material to patentability is deemed to be satisfied if all 
information known to be material to patentability of any claim issued in patent was cited by the Office 
or submitted to the Office in the manner prescribed by §§ 1.97(b)-(d) and 1.98. However, no patent will 
be granted on an application in connection with which fraud on the Office was practiced or attempted 
or the duty of disclosure was violated through bad faith or intentional misconduct. The Office 
encourages applicants to carefully examine: 

( 1 ) prior art cited in search reports of a foreign patent office in a counterpart application, 

and 

(2) the closest information over which individuals associated with the filing or 
prosecution of a patent application believe any pending claim patentably defines, to 
make sure that any material information contained therein is disclosed to the Office. 

Title 35. U.S. Code $ 101 
Inventions patentable 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Title 35 U.S. Code § 102 

Conditions for patentability; novelty and loss of right to patent 
A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a 
printed publication in this or a foreign country, before the invention thereof by the applicant for patent, 

(b) the invention was patented or described in a printed publication in this or foreign country or in 
public use or on sale in this country, more than one year prior to the date of application for patent in the United 
States, or 
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(c) he has abandoned the invention, or 

(d) the invention was first patented or caused to be patented, or was the subject of an inventor's 
certificate, by the applicant or his legal representatives or assigns in a foreign country prior to the date of the 
application for patent in this country on an application for patent or inventor's certificate field more than twelve 
months before the filing of the application in the United States, or 

(e) the invention was described in a patent granted on an application for patent by another filed in 
the United States before the invention thereof by the applicant for patent, or on an international application by 
another who has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before 
the invention thereof by the applicant for patent, or 

(f) he did not himself invent the subject matter sought to be patented, or 

(g) before the applicant's invention thereof the invention was made in this country by another had 
not abandoned, suppressed, or concealed it. In deterrnining priority of invention there shall be considered not 
only the respective dates of conception and reduction to practice of the invention, but also the reasonable 
diligence of one who was first to conceive and last to reduce to practice, from a time prior to conception by the 
other . . . 

Title 35. U.S. Code S 103 

Conditions for patentability; non-obvious subject matter 

A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art 
are such that the subject matter as a whole would have been obvious at the time the invention was made to a 
person having ordinary skill in the art to which said matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Subject matter developed by another person, which qualifies as prior art only under subsection (f) or 
(g) of section 102 of this title, shall not preclude patentability under this section where the subject matter and the 
claimed invention were, at the time the invention was made, owned by the same person or subject to an 
obligation of assignment to the same person. 

Title 35. U.S. Code, S 112 fin part) 
Specification 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise and exact terms also enable any person skilled in the art to 
which it pertains, or with which it is mostly nearly connected, to make and use the same, and shall set forth the 
best mode contemplated by the inventor of carrying out his invention. 

Title 35. U.S. Code § 119 

Benefit of earlier filing date in foreign country; right of priority 

An application for patent for an invention filed in this country by any person who has, or whose legal 
representatives or assigns have, previously regularly filed an application for a patent for the same invention in a 
foreign country which affords similar privileges in the case of applications filed in the United States or to 
citizens of the United States, shall have the same effect as the same application would have if filed in this 
country on the date on which the application for patent for the same invention was first filed in such foreign 
country, if the application in this country is filed within twelve months from the earliest date on which such 
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foreign application was filed; but no patent shall be granted on any application for patent for an invention which 
had been patented or described in a printed publication in any country more than one year before the date of he 
actual filing of the application in this country, or which had been in public use or on sale in this country more 
than one year prior to such filing. 

Title 35. U.S. Code § 120 

Benefit or earlier filing date in the United States 

An application for patent for an invention disclosed in the manner provided by the first paragraph of 
section 1 12 of this title in an application previously filed in the United States, or as provided by section 363 of 
this title, which is filed by an inventor or inventors named in the previously filed application shall have the same 
effect, as to such invention, as though filed on the date of the prior application, if filed before the patenting or 
abandonment of or termination of proceedings on the first application or an application similarly entitled to the 
benefit of the filing date of the first application and if it contains or is amended to contain a specific reference to 
the earlier filed application. 

Please read carefully before signing the Declaration attached to the accompanying Application. 
If you have any questions, please contact Morgan & Finnegan, L.L.P. 
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